Image input system

ABSTRACT

This invention provides an image input system capable of setting an arbitrary imaging inhibited area. This system displays a map showing the positions of cameras arranged in an office and the directions in which these cameras can be aimed, or displays the view angles of the cameras. An operator designates an imaging inhibited area on the map. In accordance with the designated imaging inhibited area, a limiting direction is calculated for each camera, and each camera is so controlled as not to point in that direction. Even when cameras are fixed, the system controls display so that an imaging inhibited area is not displayed.

This application is a divisional of application Ser. No. 10/865,729,filed Jun. 10, 2004, which is a continuation of Ser. No. 08/603,611,filed Jul. 25, 2000, which is a Continued Prosecution Application ofapplication Ser. No. 08/603,611 filed on Dec. 16, 1998, which is aContinued Prosecution Application of application Ser. No. 08/603,611filed on Feb. 21, 1996, to which priority under 35 U.S.C. §120 isclaimed and which are hereby incorporated by reference herein in theirentirety as if fully set forth herein.

BACKGROUND OF THE INVENTION

The present invention relates to a camera control system and, moreparticularly, to a camera control system in which at least one operatorremotely operates at least one camera.

A video transmission system has been proposed in which large numbers ofcameras and computers are arranged in a building and connected in anetwork to enable transmission of images or conversations using imagesand voices between two (or three or more) arbitrary points. For example,this system is expected to be used as a video conference system or aremote monitoring system. For these purposes, cameras whose directionand zooming can be externally controlled are already commerciallyavailable. In such a video transmission system each operator canremotely operate a given camera from a given place and/or display animage taken by a given camera on the screen of his or her computer.

In a video transmission system in which remote control of a given camerais possible, as the number of cameras to be operated increases it ismore and more necessary to allow each operator to readily know thelocation of each camera. The present applicant has already proposed asystem in which camera symbols indicating individual cameras aresuperposed on a map image which shows the installation sites of thesecameras. The present applicant has also proposed a system in which thedirection of each camera is indicated by the direction of acorresponding camera symbol.

The following problem arises when further improvements of a cameracontrol system of this sort are attempted. That is, any user using thesystem can freely remotely operate cameras connected to other hosts andsee images taken by these cameras. However, persons in the sensingenable ranges of the cameras to be remotely operated feel as if theywere being monitored, and this may give them a strong unpleasantfeeling. That is, under this condition the privacy of each person in theimaging range of a camera is invaded.

SUMMARY OF THE INVENTION

It is an object of the present invention to solve the above conventionalproblem and provide a camera control system in which the protection ofprivacy is taken into consideration.

That is, it is an object of the present invention to provide a cameracontrol system in which a range within which imaging is not permitted,i.e., an imaging inhibited range, can be set for each camera, and animage in this imaging inhibited range cannot be physically oressentially taken by a remote operation.

It is another object of the present invention to provide a cameracontrol system in which each operator can easily recognize the imaginginhibited range of each camera which he or she intends to remotelyoperate.

It is still another object of the present invention to provide a cameracontrol system which can rapidly set the imaging direction or themagnification of an image.

To achieve the above objects, a camera control system of the presentinvention has the following arrangements.

An image input system comprising:

image pick-up means for picking-up an image in a predetermined area;

setting means for setting an imaging inhibited area in which imaging isinhibited in the predetermined area;

input means for inputting an image of a desired area excluding theimaging inhibited area set by the setting means from the predeterminedarea picked-up by the pick-up means; and

display means for displaying the input image from the input means.

An image input system for displaying an input image, comprising:

at least one camera for picking-up an image of an object;

camera moving means for moving the camera by one or both of panning andtilting;

map display means for displaying a map showing a position of the cameraand a direction in which the camera can be aimed by the camera movingmeans;

setting means for setting an imaging inhibited area in whichimage-picking-up is inhibited on the map displayed by the map displaymeans; and

display means for displaying an image picked-up by a desired at leastone of the at least one camera.

An image input system for displaying an input image, comprising:

at least one camera for picking-up an image of an object;

camera moving means for moving the camera within a predetermined rangeby one or both of panning and tilting;

direction designating means for designating a direction of a desiredcamera of the at least one camera;

message display means for displaying a message indicating that thecamera cannot be moved, when the direction designated by the directiondesignating means falls outside the predetermined range; and

image display means for displaying an image picked-up by the desiredcamera.

An image input system for displaying an input image, comprising:

at least one camera for picking-up an image of an object;

camera moving means for moving the camera within a predetermined rangeby one or both of panning and tilting;

direction designating means for designating a direction of a desiredcamera of the at least one camera;

control means for aiming the camera at a position closest to thedirection designated by the direction designating means, when thedirection designated by the direction designating means falls outsidethe predetermined range; and

image display means for displaying an image picked-up by the desiredcamera.

An image input system for displaying an input image, comprising:

at least one camera for picking-up an image of an object;

region designating means for designating a partial region having adesired size and a desired position from an input image picked up by adesired camera of the at least one camera; and

display means for displaying the partial region designated by the regiondesignating means.

An image display method of displaying an image picked-up by imagepick-up means, comprising:

a setting step of setting an imaging inhibited area in which imaging isinhibited in a predetermined area;

an input step of inputting an image of a desired area excluding theimaging inhibited area set by the setting step from the predeterminedarea picked-up by the pick-up means; and

a display step of displaying the input image from the input step.

An image display method of displaying an image picked-up by at least onecamera moved by one or both of panning and tilting by camera movingmeans, comprising:

a map display step of displaying a map showing a position of the cameraand a direction in which the camera can be aimed by the camera movingmeans;

a setting step of setting an imaging inhibited area in which imagepicking-up is inhibited on the map displayed by the map display step;and

a display step of displaying an image picked-up by a desired one of theat least one camera.

An image display method of displaying an image picked-up by at least onecamera moved within a predetermined range by one or both of panning andtilting by camera moving means, comprising:

a direction designating step of designating a direction of a desiredcamera of the at least one camera;

a message display step of displaying a message indicating that thecamera cannot be moved, when the direction designated by the directiondesignating step falls outside the predetermined range; and

an image display step of displaying an image picked-up by the desiredcamera.

An image display method of displaying an image picked-up by at least onecamera moved within a predetermined range by one or both of panning andtilting by camera moving means, comprising:

a direction designating step of designating a direction of a desiredcamera of the at least one camera;

a control step of aiming the camera at a position closest to thedirection designated by the direction designating step, when thedirection designated by the direction designating step falls outside thepredetermined range; and

an image display step of displaying an image taken by the desiredcamera.

An image display method of displaying an image picked-up by at least onecamera, comprising:

the region designating step of designating a partial region having adesired size and a desired position from an input image taken by adesired camera of the at least one camera; and

the display step of displaying the partial region designated by theregion designating step.

A computer readable memory storing a program of displaying an imagetaken by image pick-up means, comprising:

codes of a process of setting an imaging inhibited area in which imagingis inhibited in a predetermined area;

codes of a process of inputting an image of a desired area excluding theimaging inhibited area set by the setting step from the predeterminedarea; and

codes of the display step of displaying the input image from the inputstep.

A computer readable memory of the present invention has the followingarrangements.

A computer readable memory storing a program of displaying an imagetaken by at least one camera moved by one or both of panning and tiltingby camera moving means, comprising:

codes of a process of displaying a map showing a position of the cameraand a direction in which the camera can be aimed by the camera movingmeans;

codes of a process of setting an imaging inhibited area in which imagingis inhibited on the map displayed by the map display step; and

codes of a process of displaying an image taken by a desired one of theat least one camera.

A computer readable memory storing a program of displaying an imagetaken by at least one camera moved within a predetermined range by oneor both of panning and tilting by camera moving means, comprising:

codes of a process of designating a direction of a desired camera of theat least one camera;

codes of a process of displaying a message indicating that the cameracannot be moved, when the direction designated by the directiondesignating step falls outside the predetermined range; and

codes of a process of displaying an image taken by the desired camera.

A computer readable memory storing a program of displaying an imagetaken by at least one camera moved within a predetermined range by oneor both of panning and tilting by camera moving means, comprising:

codes of a process of designating a direction of a desired camera of theat least one camera;

codes of a process of aiming the camera at a position closest to thedirection designated by the direction designating step, when thedirection designated by the direction designating step falls outside thepredetermined range; and

codes of a process of displaying an image taken by the desired camera.

The camera control system and method of the present invention with theabove arrangements can designate an imaging inhibited area and therebyprotect the privacy of a user. Additionally, since the imaging inhibitedarea can be set in a map window, the operation is easy and the imaginginhibited area can be clearly designated.

Also, even if a direction in which a camera is to be aimed is outsidethe movable range of the camera, the movement of the camera isrestricted within the camera movable range. This prevents a damage tothe camera.

Furthermore, an operator can set an imaging inhibited area and designatethe direction of a camera while monitoring the view angle of the camera.This facilitates the operation.

The directions or the magnifications of a camera can be instantaneouslyswitched.

Images such as are obtainable when a plurality of cameras are used canbe obtained by using a single camera physically.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a schematic block diagram showing the basic configuration of acomputer system according to one embodiment of the present invention;

FIG. 2 is a schematic block diagram showing the hardware configurationof the embodiment in a network;

FIG. 3 is a schematic block diagram showing the software configurationof the embodiment in a network;

FIG. 4 is a view showing an example of a camera display control paneldisplayed on a display 35 by a camera control client 56;

FIG. 5 is a view for explaining setting of an imaging inhibited area;

FIG. 6A is a flow chart showing processing of setting an imaginginhibited area;

FIG. 6B is a flow chart showing the basic operation of a cameramanagement server 50 with respect to the setting of an imaging inhibitedarea;

FIG. 7 is a detailed flow chart of step S2 in FIG. 6B;

FIG. 8 is a schematic view showing the correspondence between an imaginginhibited area and an imaging inhibited direction;

FIG. 9 is a view showing an example of a camera status table;

FIGS. 10A and 10B are schematic views showing the relationship betweenthe pan enable direction and the imaging inhibited direction in case 1;

FIG. 11 is a schematic view showing the relationship between the panenable direction and the imaging inhibited direction in case 2;

FIG. 12 is a schematic view showing the relationship between the panenable direction and the imaging inhibited direction in case 3;

FIG. 13 is a schematic view showing the relationship between the panenable direction and the imaging inhibited direction in case 4;

FIG. 14 is a schematic view showing the relationship between the panenable direction and the imaging inhibited direction in case 5;

FIG. 15 is a detailed flow chart of step S15 in FIG. 7;

FIG. 16 is a view showing an example of a permitted host informationfile;

FIG. 17 is a view for explaining another method of setting an imaginginhibited area;

FIG. 18 is a view showing an example of a file for connecting setimaging inhibited areas and objective cameras;

FIG. 19 is a block diagram showing a common configuration applied to thesecond to fourth embodiments of a video camera controller according tothe present invention;

FIG. 20 is a view showing a common graphic user interface applied to thesecond to fourth embodiments;

FIG. 21 is a view showing a common video camera control screen appliedto the second to fourth embodiments and particularly explaining thesecond embodiment;

FIG. 22 is a view showing an example of determination of the cameramovable range;

FIG. 23 is a flow chart showing the operation of the second embodiment;

FIGS. 24A and 24B are views showing an outline of the operation of thethird embodiment;

FIG. 25 is a view for explaining the operation of the third embodiment;

FIG. 26 is a flow chart for explaining the operation of the thirdembodiment;

FIG. 27 is a view showing an outline of the operation of the fourthembodiment;

FIG. 28 is a view for explaining the operation of the fourth embodiment;

FIG. 29 is a flow chart for explaining the operation of the fourthembodiment;

FIG. 30 is a block diagram showing an outline of the configuration ofthe fifth embodiment of the present invention;

FIG. 31 is a view showing an example of the screen in the fifthembodiment;

FIG. 32 is a view for explaining an input image and an extraction range;

FIG. 33 is a flow chart of the fifth embodiment;

FIG. 34 is a view showing an example of the screen when four extractionranges are set;

FIG. 35 is a view for explaining four extraction ranges set in an inputimage;

FIG. 36 is a view showing a correspondence table of the extractionranges and camera windows;

FIG. 37 is a flow chart for explaining an operation when N virtualcameras are included in a system; and

FIG. 38 is a view showing an example of a user interface for operatingand displaying the extraction ranges.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described below withreference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing an outline of the configuration of acomputer system in which a plurality of video communication terminals,i.e., cameras, which are basic elements in this embodiment of thepresent invention, are connected. One or more computers having theconfiguration shown in FIG. 1 and/or one or more computers having asimilar configuration are interconnected with each other via a computernetwork.

In FIG. 1, camera control circuits 12 (12-1, 12-2, 12-3, . . . )directly control, e.g., panning, tilting, zooming, focus adjustment, andaperture of video cameras 10 (10-1, 10-2, 10-3, . . . ) in accordancewith external control signals. A camera input selector 14 selects avideo camera 10 to be controlled and inputs an output signal from thecamera. RS-232C is an example of a control signal line. However, thepresent invention is obviously not limited to this interface. Note thatalthough an output signal is usually a video signal, a video signal andan audio signal are output if a camera with a microphone is used. In thefollowing description it is assumed that only a video signal is output.

A video communication terminal 20 sends a control command to a desiredcamera control circuit 12 via the camera input selector 14 and controlsa video camera 10 connected to the camera control circuit 12. Also, thevideo communication terminal 20 transmits an image picked-up by theselected camera to the network and receives images from the network. ACPU 22 controls the overall system by executing programs stored in amain storage 24 and a secondary storage (e.g., a hard disk drive) 26. Amouse 28 is a pointing device. The mouse 28 and a keyboard 30 are usedby an operator as input devices.

An I/O port 32 connects the camera input selector 14 and supplies cameracontrol commands to the camera input selector 14. A video board 34inputs an output video signal from a video camera 10 selected by thecamera input selector 14 and displays various images on a bit mapdisplay 35. A network interface 36 connects the video communicationterminal 20 to a computer network or a communication network. A systembus 38 interconnects the devices from the CPU 22 to the networkinterface 36. Through the network interface 36 it is possible to send acamera control signal from a remote place to the video communicationterminal 20 via the network and thereby control a given camera 10.

The camera input selector 14 selects one of the control signal lines andone of the video outputs connected to the camera control circuits 12.The camera input selector 14 supplies the selected video output to thevideo board 34 and logically connects the selected control signal lineto the I/O port 32. An NTSC signal which is a luminance/color-differenceseparation type signal is an example of the video signal format. Thevideo board 34 inputs the video output selected by the camera inputselector 14. The input video signal is displayed as a dynamic image in apredetermined window of the bit map display 35 and/or transmitted toanother terminal.

The secondary storage 26 stores various information, e.g., cameraposition information data and camera graphic data, pertaining to thecameras 10 and other cameras connected via the network. Details of thesepieces of information will be described later.

If only one camera 10 is connected, the camera input selector 14 isunnecessary and one camera control circuit 12 is directly connected tothe I/O port 32. Also, if no video signals are transmitted, the cameras10, the camera control circuits 12, and the camera input selector 14 areunnecessary.

The apparatus shown in FIG. 1 is connected as a communication terminalto a network as illustrated in FIG. 2. Not all communication terminalsneed to have the configuration shown in FIG. 1. For example, the networkcan include a communication terminal to which only one camera isconnected or a terminal to which no camera is connected (i.e., aterminal having a function of only remotely operating cameras connectedto other terminals and displaying images from these cameras). Generally,communication terminals having such various configurationssimultaneously exist in a single network. As the network used in thisembodiment, a LAN or a WAN having an enough transmission band width totransmit digital dynamic image data and camera control signals isassumed. Dynamic image data is usually compressed before beingtransmitted. In this embodiment, however, a detailed description of thedynamic image compression method will be omitted since various existingmethods are usable.

The video board 34 has the video capture function as described above.The video board 34 supplies input video data to the bit map display 35to display images on it and also supplies the data to the CPU 22 throughthe bus 38. The CPU 22 packetizes the video data and outputs the packetto the network via the network interface 36. Packets of, e.g., a cameraoperation instruction and a camera switch instruction also are sent fromthe network interface 36 to the network. Additionally, packets ofinformation pertaining to the overall system are sent to the network.These pieces of information are transmitted to either specifiedterminals or all terminals in accordance with the contents of data to betransmitted and with the necessity.

Reception is done in a similar fashion. That is, upon receiving packetsof video data, a camera operation instruction, and a camera switchinstruction, each video communication terminal 20 processes the receivedvideo data in the same manner as for internal capture data and processesthe received camera operation and camera switch instructions in the sameway as for analogous internal instructions. The information pertainingto the overall system is used to update system display of a userinterface (to be described later).

FIG. 3 is a block diagram showing the software configuration in thesystem of this embodiment. Referring to FIG. 3, the video communicationterminals 20 and a camera management server 50 are connected to acomputer network 52. A camera control server 54, a camera control client56, and image transmitting/receiving software 58 are installed in eachvideo communication terminal 20. The camera control server 54 controlsthe camera 10 in accordance with a camera control signal (including aselect signal if a plurality of cameras are connected) entered from theself-terminal or transmitted from another terminal. The camera controlclient 56 remotely operates the camera 10 of the self-terminal oranother terminal. The image transmitting/receiving software 58 suppliesan image from the camera 10 of the self-terminal to other terminals viathe network 52 and supplies images transferred from other terminals viathe network 52 and an image from the camera 10 of the self-terminal tothe display 35 of the self-terminal.

The camera management server 50 is software for managing all cameras 10available (or connected) to the network 52 and holds information such asthe camera name, host name, installation position, and current status ofeach camera. The camera management server 50 also manages registrationof a camera which is newly made usable via the network 52 and delete ofa camera disconnected from the network 52. Additionally, the cameramanagement server 50 periodically informs all camera control clients 56of the management information of all cameras. The camera managementserver 50 need only be stored in one of the terminals connected to thenetwork 52.

The camera control client 56 displays the locations and directions ofthe camera 10 of the self-terminal and all cameras usable via thenetwork 52 on the screen of the display 35 of the self-terminal bysuperposing predetermined camera symbols on a map. The camera controlclient 56 also updates the display state of each camera symbol in realtime on the basis of the camera information periodically sent from thecamera management server 50.

The camera control server 54, the cameral control client 56, the cameraimage transmitting/receiving software 58, and the camera managementserver 50 are realized when the CPU 22 executes respective predeterminedprograms.

FIG. 4 shows an example of a camera display control panel displayed onthe display 35 by the camera control client 56. On a map showing theinstallation locations of operable cameras, a map window 60 superposescamera icons indicating the locations and directions of these cameras. Acamera image window 62 displays an image from one selected camera. Acamera operation panel 64 includes various camera control buttons andcontrols panning, tilting, and zooming of a selected camera. In thisembodiment, it is assumed that a window display system capable ofsimultaneously displaying a plurality of windows is operating.

The map window 60 displays a map showing, e.g., the arrangement of seatsin an office. On this map camera icons 66 indicative of the locations ofindividual cameras arranged in the office are superposed. Each cameraicon 66 is displayed in the same position as the location of thecorresponding camera and points in almost the same direction as thecurrent camera direction. These camera icons 66 are displayed indifferent colors to distinguish between a camera which is currentlyselected to display an image or to be remotely operated, cameras used byother users, and cameras not used by anybody.

The operation panel 64 is displayed below the camera image window 62.The operation panel 64 includes rotation operating buttons for panningand tilting and two buttons for a zoom operation. By operating thesebuttons, a user can operate rotation (panning and tilting) and zoomingof a given designated camera. If an operation of a selected camera isimpossible (e.g., if the camera is already being operated by some otheruser), the rotation operating buttons and the zoom operating buttons aredisplayed in an operation disable state.

For instance, when a certain user desires an access to (in this case aremote operation of) a certain camera, the user double-clicks the cameraicon indicating the camera of interest. In accordance with thisdouble-click, the camera control client 56 requests the cameramanagement server 50 to issue the access right to that camera. If thereis no user currently operating that camera, the camera management server50 grants a permission to perform a remote operation (including displayof an image) of the camera. If there is a user operating the camera, thecamera management server 50 denies the access right. If the access rightis granted, an output image from the camera is displayed in the cameraimage window 62 and operations (panning, tilting, and zooming) from thecamera operation panel 64 are enabled.

Below the operation panel 64, a set button 68 for setting an imaginginhibited area and a clear button 70 for canceling the set imaginginhibited area are displayed. In this embodiment, sensing a scene of animaging inhibited area by the cameras is inhibited. The imaginginhibited area can be set in a camera control system which remotelyoperates a desired camera within an entire range in which the camera isoperable. The imaging inhibited area is set for designated cameras orall cameras under the camera control system. A camera for which animaging inhibited area is set cannot be remotely operated via thenetwork to point in a direction in which the imaging inhibited area issensed.

The processing of setting an imaging inhibited area in this embodimentwill be described in detail below.

When a user clicks the imaging inhibited area set button 68, the cameracontrol client 56 draws vertical and horizontal lines partitioning themap window 60 into a mesh as illustrated in FIG. 5. Areas divided bythese vertical and horizontal lines are set units of an imaginginhibited area. On the map window 60, the user designates one or moredivided areas in which he or she wants to inhibit sensing. Thedesignated divided area (e.g., an area 92 in FIG. 5) is enclosed by,e.g., red so as to stand out clearly from other divided areas. When adesired divided area is selected as an imaging inhibited area, the userdouble-clicks the mouse button on the selected divided area.Consequently, an imaging inhibited area is set. To cancel the imaginginhibited area, it is only necessary to click the clear button 70 byusing the mouse.

When the imaging inhibited area is set or changed, this information issupplied to the camera management server 50. The camera managementserver 50 changes the direction of a camera whose sensing range (fieldof view) includes the set or changed imaging inhibited area so that theimaging inhibited area is not sensed. Also, the direction of thecorresponding camera icon 66 is similarly changed on the map window 60.That is, the camera control client 56 operating in the communicationterminal 20 in which the imaging inhibited area is set informs thecamera management server 50 that the imaging inhibited area is set, andsends coordinate information of the set imaging inhibited area to theserver 50.

FIG. 6A is a flow chart of the processing of the camera control clientwhen an imaging inhibited area is set. This setting is done as describedabove.

FIG. 6B is a basic flow chart of the camera management server 50 withrespect to the setting and change of an imaging inhibited area. Thecamera management server 50 acquires coordinate information (coordinatesof opposite corners of a rectangle) of the set imaging inhibited area(step S1). The camera management server 50 searches for a camera whosecurrent sensing range includes even a portion of the set imaginginhibited area. The camera management server 50 issues to thecorresponding camera a camera control command for changing the directionof the camera to a direction in which the imaging inhibited area movesout of the sensing range, thereby changing the imaging direction of thecamera (step S2). Also, the camera management server 50 informs allcamera control clients 56 of the changed direction of the camera whosedirection is thus changed, and updates the direction of thecorresponding camera icon 66 on the map window 60 (step S3).

FIG. 7 shows a detailed flow chart of step S2 in FIG. 6B. First, acamera whose direction is to be changed and a camera whose rotatableangle is to be changed are selected (step S11). Assume that cameras in aself-station locally connected to a host in which the imaging inhibitedarea is set are objective cameras. The camera management server selectsthese cameras by referring to host information of each camera.

The direction and the rotatable angle of each selected camera arechanged. That is, of the cameras selected in step S11, a camera i ofinterest (whose zoom lens is set to the telephoto side) is selected instep S12. From the coordinates of the installation position of thecamera i and the coordinates of the imaging inhibited area, an imaginginhibited angle θd (d<θd<D) in case that the optical system of thecamera i is set to the widest angle is calculated as illustrated in FIG.8 (step S13). Angles d and D are centered around the camera i at the twoends of the imaging inhibited area on the map plane. Camera control ofthe camera i is temporarily locked so that no user operates the camera ibefore the camera i is completely updated (step S14). From a camerastatus table (to be described later) and the imaging inhibited anglecalculated in step S13, the pan enable angle of the camera i is changedso that the camera i does not aim in the imaging inhibited direction. Ifthe camera i is currently pointing in the imaging inhibited direction, acamera control command which changes the direction of the camera so thatthe camera moves out of the imaging inhibited angle is transmitted to acorresponding camera control server 54 (step S15). Details of thischange processing will be described later. When the necessary changeprocessing is completed, camera control lock of the camera i is released(step S16).

The processing from step S13 to step S15 is executed for all of thecameras selected in step S11 by changing the camera of interest in stepsS17 and S18.

FIG. 9 shows an example of a camera status table 901 storing fixedinformation and current information of each individual camera. The table901 consists of information such as camera numbers, camera names, hostnames, the coordinates of the camera installation positions on the map,initial directions upon start-up, pan enable angles θp. (a<θp<A), andcurrent directions. The two ends of the pan enable angle are angles aand A, similar to d and D, represented by clockwise angles in adirection x (a predetermined direction on the map plane).

The camera management server 50 constantly checks the camera statustable and so monitors that the individual cameras do not rotate beyondthe respective pan enable angles. Also, the camera management server 50periodically sends the contents of the camera status table to the cameracontrol clients 56. On the basis of the information from the cameramanagement server 50, each camera control client 56 updates thedirection of each camera icon 66 in the map window 60 displayed on thedisplay 35.

The relationships between the pan enable angle θp (a<θp<A) and theimaging inhibited angle θd (d<θd<D) are classified into five casesdescribed below.

In case 1, A>a>D>d or a<A<d<D as illustrated in FIGS. 10A and 10B. Inthis case the pan enable angle and the imaging inhibited angle do notoverlap at all.

In case 2, d<a<D<A as shown in FIG. 11. In this case a portion of theimaging inhibited angle overlaps the pan enable angle.

In case 3, a<d<D<A as shown in FIG. 12. In this case the imaginginhibited angle is entirely contained in the pan enable angle.

In case 4, a<d<A<D as shown in FIG. 13. In this case a portion of theimaging inhibited angle overlaps the pan enable angle.

In case 5, d<a<A<D as shown in FIG. 14. In this case the pan enableangle is entirely contained in the imaging inhibited angle.

FIG. 15 shows the details of step S15 in FIG. 7. The camera status table901 and the direction of the camera of interest are changed as followsin accordance with the five cases described above. Assume the currentdirection of the camera of interest is N.

First, in step S21 which of the five cases is the current case ischecked.

If case 1 is determined, this means that the imaging inhibited angle andthe pan enable angle do not overlap each other. Accordingly, no changeis made (step S22).

If case 5 is determined, this means that the pan enable angle isentirely contained in the imaging inhibited angle. Therefore, controland display of the camera are turned off (step S23).

If case 2 is determined, the current direction N of the camera ischecked (step S24). If a<N<D is determined (YES in step S25), this meansthat the camera is currently aiming in the imaging inhibited area, andso the direction of the camera also needs to be changed. Accordingly, inthe camera status table 901 the angle D outside the imaging inhibitedangle is substituted into the information of the current cameradirection N and into the end portion a of the pan enable angle (stepS26). Thereafter, a pan command for aiming the camera in the angle D isissued (step S27).

Cases 3 and 4 are basically the same as case 2. In these cases, the panenable angle is so changed as not to contain the imaging inhibitedangle, and the direction of a camera pointing to the imaging inhibitedangle is so changed that the camera does not aim at the imaginginhibited angle (steps S28 to S36). If a camera is pointing to theimaging inhibited angle in case 3, the camera is aimed at the angle D.However, it is also possible to point the camera to the angle d or to alarger end portion by comparing (A−D) with (d−a).

In the above embodiment, the imaging inhibited area is chosen fromequally divided areas on the map. However, it is evident that arectangle with a given size drawn on the map by dragging the mouse canalso be set as an imaging inhibited area. In this case each user can setan imaging inhibited area with a given size.

In the above embodiment, after the imaging inhibited area is set, onlycameras connected to the local host are chosen as a camera whose panenable angle and direction of the cameras are to be changed.Accordingly, it is possible to impose limitations on the directions ofthe cameras in the self-station. Consequently, other users cannotremotely aim the cameras in the self-station in the direction of an areaset as the imaging inhibited area. However, if only local cameras arethe objects, the number of cameras whose camera operations can berestricted is limited. In situations where a large number of camerasexist, therefore, even if an imaging inhibited area is set it issometimes possible to take an image of this imaging inhibited area froma camera of another host. Also, it is sometimes necessary to limit theoperation of a certain camera from other hosts rather than a local host.

This objective or demand can be achieved by making an extension suchthat cameras connected to other hosts can also be set as a camera whosepan enable direction and imaging direction are changed in accordancewith setting (or change) of an imaging inhibited area. This extension ismade as follows.

Camera access permitted hosts are predetermined for each camera. As anexample, a permitted host information file as shown in FIG. 16 isformed. If an imaging inhibited area is set on the map of host 1, forexample, all cameras connected to all hosts to which a remote operationby host 1 is permitted are retrieved from the permitted host informationfile and set as objects to be changed. If all host names are set for acertain camera, it is of course possible to inhibit any camera frombeing aimed at the imaging inhibited area. If this is the case, theimaging inhibited area can never be imaged.

It is unnecessary to select an imaging inhibited area from equallydivided areas on the map. For example, as illustrated in FIG. 17, it ispossible to preset several areas with arbitrary sizes which may be setas an imaging inhibited area, store the position information of theseareas, and, when an imaging inhibited area is to be set (changed),display these areas on the map so that any area can be selected.Accordingly, areas which can be set as an imaging inhibited area andareas which cannot can be distinguished in accordance with the user. Asan example, a common place which is strongly public cannot be set as animaging inhibited area when an imaging inhibited area is set (changed),if the place is not set as a candidate of an imaging inhibited area.

It is also possible to limit cameras (or hosts) whose operations arelimited by an imaging inhibited area for each set imaging inhibitedarea. If this is the case, as illustrated in FIG. 18, a file connectingobjective cameras to each set imaging inhibited area is formed. In FIG.18, if area 1 is chosen as an imaging inhibited area, cameras 1, 2, 3,and 5 are set as cameras whose pan is limited. This allows a rapidretrieval of cameras covering the imaging inhibited area.

In the above description, general users can freely set and cancel animaging inhibited area. However, it is naturally also possible to permitonly a specific manager to do these works. For example, a system manageror a direct manager of an objective camera (e.g., the user of a videocommunication terminal connected to the camera) can set an imaginginhibited area of each camera under the management and can change andcancel the setting. However, general users cannot set imaging inhibitedareas of cameras which they do not manage and cannot change and cancelalready set imaging inhibited areas.

As can be readily understood from the above explanation, in the presentinvention an imaging inhibited area which is inhibited from being imagedby a camera can be set on a map showing the arrangement of cameras. Thismakes it possible to protect the privacy of each user. Also, anoperation for this purpose can be done by a readily understandableinterface.

Second Embodiment

The second embodiment of the present invention will be described indetail below with reference to the accompanying drawings.

FIG. 19 is a block diagram showing a video communication terminal ofthis embodiment. A display 100 as a display unit displays a video cameracontrol window 10, FIG. 20. A video camera 101 incorporating a zoom lenscan be panned and tilted by a panhead 109 which incorporates a motor(not shown). The motion of the video camera 101 is controlled by a videocamera controller 102. An image controller 103 encodes and decodes imagesignals. A communication controller 104 exchanges image information andvideo camera control information with a video camera controllerinstalled in a remote place. A system controller 105 controls theoverall system. An operation unit 106 as a designating means includes akeyboard, a mouse, and a digitizer and inputs designations to the systemcontroller 105. These controllers constitute a control means accordingto this embodiment.

The configuration in FIG. 19 is equivalent to setting one video cameraand omitting the camera input selector 14 in the configuration inFIG. 1. That is, the system controller 105, the video camera controller102, the image controller 103, and the communication controller 104correspond to the video communication terminal 20, the camera controlcircuit 12, the video board 34, and the network I/F 36, respectively.Accordingly, a plurality of cameras can also be controlled by adding thecamera input selector 14 to the configuration in FIG. 19.

FIG. 20 shows a graphical user interface (GUI) on a screen 100 a of thedisplay 100.

A video camera control window 110 includes an image display screen 111,a video camera control screen 112, camera direction control buttons 113,114, 115, and 116, and sensing range (field of view) control buttons 117and 118. The image display screen 111 displays an image currently beingpicked-up by the video camera 101. The video camera control screen 112is used to control the motion of the video camera 101. The cameradirection control buttons 113, 114, 115, and 116 are used to change thecamera direction upward, to the left, downward, and to the right,respectively, without using the video camera control screen 112. Thesensing range control buttons 117 and 118 are used to enlarge and reduce(zoom in and zoom out), respectively, an image of an object. The controlwindow 110 also displays a cursor 119 for operating the control window110. This cursor 119 is operated by the operation unit 106 (FIG. 1). Thecontrol window 110 further includes a button 107 for designating animaging inhibited range and a clear button 108 for canceling the rangedesignated by the button 107.

FIG. 21 is an enlarged view of the video camera control screen 112.Referring to FIG. 21, a sensing enable area 112 a indicates a rangewhich can be sensed by the video camera 101 when the zoom magnificationis minimized (to the widest angle) and panning and tilting are performedover the entire range. The sensing enable area 112 a has the same sizeas the video camera control screen 112. A video camera movable area 112b indicates a range within which the optical axis of the video camera101 can move. A sensing range display area 112 c indicates a positionand a range in the sensing enable area 112 a which the video camera 101is currently sensing. Although the size of the sensing enable area 112 ais fixed, the sensing range display area 112 c changes its size inaccordance with the zoom magnification.

The video camera control screen 112 is displayed in place of the mapdisplay window 60 in the first embodiment. In the apparatus of thisembodiment, it is assumed that one camera is connected to one videocommunication terminal. Accordingly, by displaying the sensing anglebased on the visual field of the connected camera, the sensing enablerange and the camera rotatable angle can be shown to the user.

Designation of an imaging inhibited range is done by the button 107. Thebutton 107 has the same function as the button 68 in the firstembodiment. When the button 107 is depressed, the map window 60, FIG. 5,in which the arrangement of cameras and already divided areas are drawnis displayed, and an imaging inhibited area is designated on this map.In the same fashion as in the first embodiment, an angle at whichcameras are inhibited from being aimed is calculated as shown in FIG. 8and registered as a camera status table 901, FIG. 9. Although the tablein FIG. 9 does not contain a camera tilt angle, in this embodiment it isassumed that the camera status table contains the tilt angle.

When the imaging inhibited area is thus designated, an angle at which acamera is rotatable is displayed on the camera control screen 112, inplace of the map window 60, on the basis of the contents of the camerastatus table. In this case the camera movable area 112 a is determinedon the basis of a pan enable angle (a, A) registered in the camerastatus table 901. Note that the pan enable angle and the tilt enableangle excluding the imaging inhibited angle will be collectivelyreferred to as a camera movable range hereinafter.

FIG. 21 shows an example in which no imaging inhibited area isdesignated. If an imaging inhibited area is designated, as shown in FIG.22, a permitted camera movable range is obtained by projecting the panenable range of the camera onto a plane. This corresponds to theposition and length in the horizontal direction of the camera movablearea 112 b and determines the positions of vertical lines 112 b-1 and112 b-2 of the area 112 b.

An imaging inhibited area also can be designated on the camera controlscreen 112 in FIG. 21. That is, the camera movable area is directlydesignated by an operator by dragging the vertical line 112 b-1 or 112b-2 and moving the line to a desired position by using the mouse. Inthis case the angles (A, a) at the two ends of the pan enable angle arecalculated from the permitted camera movable area by a procedure whichis the reverse of the procedure in FIG. 22.

A control procedure performed by the system controller 105 when anoperator operates the camera on the camera control screen 112 thusobtained will be described below with reference to the flow chart inFIG. 23.

The user operates the video camera control screen 112, the cameradirection control buttons 113 to 116, or the sensing range controlbuttons 117 and 118 in the video camera control window 110 by using thecursor 19, thereby controlling the video camera 110 to take an image ofa desired location (step S230).

The coordinates of the selected camera direction are detected (stepS231). An area on the camera control screen in which the selectedcoordinates are present is checked (step S232).

In this embodiment, if the user selects a position outside the videocamera movable area on the video camera control screen 112 in FIG. 21 byusing the cursor 119 (119-1), an error is determined because theselected position is outside the video camera movable area, and nothingis done for the operation control of the video camera 101. Preferably,to inform the user that the selection is an error, a message “positionoutside camera movable area is selected” is displayed on the videocamera control screen 112 (step S234). It is also possible, withoutdisplaying this message, to inform the user that the position cannot beselected, by replacing the cursor 119-1 indicating the position outsidethe video camera optical axis movable area with a symbol indicated byreference numeral 119-2, which is a cursor with a mark “x” on it.

On the other hand, if the designated direction is within the cameramovable area, the camera is aimed in that direction, and the camerastatus table is updated (steps S233 and S237). If the object camera tobe operated is a remote-controlled camera, control information forrotating the camera is transmitted to a system control unit of theobject camera (step S236).

With the above control it is possible to prevent a damage to the panhead109 even if the selected position is outside the video camera movablearea. Also, an error message is displayed although the video camera 101does not move, so the user can instantly know that the selection is anerror.

It is readily possible to determine that the selected place is outsidethe video camera movable area, when the system controller 105 calculatesthe coordinates of the selected place and detects that the calculatedcoordinates are outside the video camera movable area 112 b and insidethe imaging enable area 112 a. More specifically, a method in the thirdembodiment is applicable. If the system controller 105 detects that theselected place is outside the video camera movable area, the messagedescribed above can be displayed on the display 100 via the imagecontroller 103.

In the above embodiment, the video camera 101 is not operated if theselected place is outside the video camera movable area. However, thefollowing effect can be obtained by operating the video cameracontroller 102 to set zooming of the video camera 101 to thewidest-angle without panning and tilting the camera. That is, the userselected the place because he or she wanted to see the place. Therefore,it is desirable that the selected place be displayed even though theplace is not displayed in the center of the screen. If the optical axisof the video camera 101 positions at the edge portion of the videocamera movable area 112 b, an image of the selected place (outside thecamera movable area, in this case) can sometimes be displayed by settingzooming of the camera to the widest-angle. If this is the case thedesire of the user is satisfied.

Third Embodiment

In this embodiment, the content in step S235 of FIG. 23 in the secondembodiment is changed. In the second embodiment, as described above,even if the place selected by the user is outside the video cameramovable area, the user selected the place because he or she wanted tosee the place. Therefore, it is desirable that the selected place bedisplayed even though the place is not displayed in the center of thescreen. In this embodiment, this desire is positively satisfied. Thisembodiment will be described in detail below with reference to theaccompanying drawings. Note that the system configuration is identicalwith that of the second embodiment shown in FIG. 19 and the GUI is alsosimilar to that of the second embodiment shown in FIG. 20, and sodetailed descriptions thereof will be omitted.

FIGS. 24A and 24B illustrate operations on a video camera control screen112. FIG. 24A shows a normal operation in which the user selects aposition inside a video camera movable area 112 b as a new object to besensed. FIG. 24B shows a case where the user selects a position outsidethe video camera movable area 112 as a new object to be sensed.

Referring to FIG. 24A, a cursor 119 is moved by an operation unit 106such as a mouse to select a predetermined position, a coordinate point120 in this case, thereby moving an imaging range display area from anarea 112 c to an area 112 d. An image in this new area 112 d isdisplayed on an image display screen 111 (FIG. 20). In this case thezoom magnification is not altered and the imaging direction of a videocamera 101 is so changed that the selected coordinate point 120 isdisplayed in the center of the screen.

In FIG. 24B, as in the case of FIG. 24A, the sensing range display areais moved from the area 112 c to the area 112 d by selecting thecoordinate point 120. Since the selected coordinate point 120 is outsidethe video camera movable area 112 b, it is not possible to change thesensing direction of the video camera 101 toward that position.Therefore, an intersection 123 of a straight line 122 connecting acentral coordinate point 121 of an imaging enable area 112 a and theselected coordinate point 120 and the frame of the video camera movablearea 112 b is set as a new imaging screen center. However, if the cameradirection is changed without changing the zoom magnification, theselected coordinate point 120 (in FIG. 24B) sometimes does not come intothe sensing range as indicated by the positional relationship betweenthe sensing range display area 112 d and the selected coordinate point120. Therefore, the zoom magnification is minimized (to the widestangle). By the above control, the selected coordinate point 120 iscontained in an area 112 e broader than the sensing range display area112 d, and this makes sensing at the selected coordinate point 120possible.

Details of the operation of the video camera control method in thisthird embodiment will be described below with reference to FIGS. 25 and26. FIG. 25 illustrates the sensing enable area 112 a as a plane havinga size of m×n. Coordinates are written in this plane, and the plane isdivided into a plurality of areas. Reference numerals in FIG. 25 are thesame as in FIG. 24B. FIG. 26 is a flow chart showing the operation of asystem controller 105. Corresponding step numbers are given inparentheses.

The user operates the operation unit 106 to move the cursor 119 in avideo camera control window 110 on a screen 110 a of a display 100,selecting a place to be sensed (step S100).

The system controller 105 detects coordinates (Xp,Yp) 120 of theposition of the cursor 119 when the place to be sensed is selected instep S100 (step S101).

The system controller 105 checks which of areas A, B, C, D, and E theselected coordinate point (Xp,Yp) 120 belongs to (step S102).

If the system controller 105 determines in step S102 that the selectedcoordinate point (Xp,Yp) 120 is within the area A, i.e., the videocamera movable area 112 b, it is unnecessary to change the zoom value.Accordingly, the system controller 105 calculates the moving amount ofthe video camera (step S103).

On the other hand, if the system controller 105 determines in step S102that the selected coordinate point (Xp,Yp) 120 is in any of the areas B,C, D, and E outside the video camera movable area 112 b, the systemcontroller 105 calculates coordinates of a new sensing center 123 (stepS104). For example, if it is determined that the selected coordinatepoint (Xp,Yp) 120 is in the area B as illustrated in FIG. 25, the newsensing center (the position of the optical axis of the video camera101) 123 is calculated as follows.

That is, assuming the coordinates of the center in the sensing enablearea 112 a are (Xc,Yc), the straight line 122 passing the selectedcoordinate point (Xp,Yp) 120 and the central coordinate point (Xc,Yc)121 is given by(y−Yc)(Xp−Xc)=(x−Xc)(Yp−Yc)Since the frame of the video camera movable area 112 b in contact withthe area B isy=Ybthe coordinates of the new imaging center 123 as an intersection of thestraight line 122 and the frame of the video camera movable area 112 bare calculated by((Yb−Yc)(Xp−Xc)/(Yp−Yc)+Xc,Yb)

The system controller 105 then calculates the amount of movement to thenew sensing center 123 calculated in step S104. If an operation ofminimizing the zoom magnification (to the widest angle) is also to beperformed, the system controller 105 performs a calculation for theoperation (step S105).

Subsequently, the system controller 105 checks whether the video camera101 as an object to be controlled is connected to a video cameracontroller in a remote place or can be controlled by the controller 105(step S106).

If the system controller 105 determines in step S106 that the videocamera 101 is connected to a video camera controller in a remote place,the system controller 105 transmits the video camera control informationcalculated in step S103 or S105 to the video camera controller in theremote place via a communication controller 104 (step S107).

If the system controller 105 determines in step S106 that the videocamera 101 can be controlled by the controller 105, the systemcontroller 105 moves the camera direction and changes the zoommagnification of the video camera 101 on the basis of the controlinformation of the video camera 101 calculated in step S103 or S105(step S108).

By repeating the operation from step S100 to step S108 described above,it is possible to pick-up an image of a given object located in thesensing enable range of the video camera 101 as an object to becontrolled.

In the processing of this embodiment, interrupt is accepted. Interruptoccurs when the user completes the operation of the system-(step S109),and the processing is ended.

In this embodiment as described above, even if a place selected by theuser is outside the video camera movable area, the video camera 101 ismoved to the limit toward the position selected by the user and stoppedat that limiting position, and an image corresponding to the field ofview of the video camera 101 is displayed. Accordingly, no additionalload is applied on the camera driver and an image of the selected placeor of the selected place and its vicinity is taken (displayed).Consequently, the desire of the user to see the selected place can besatisfied. Also, the camera cannot be pointed to the designated imaginginhibited area even if the user attempts to aim the camera in thatdirection.

Furthermore, when the zoom magnification is minimized (to the widestangle), an image of the selected place can be reliably taken(displayed).

Fourth Embodiment

The fourth embodiment is obtained by further improving the thirdembodiment. This fourth embodiment will be described below withreference to the accompanying drawings. The configuration, GUI, andvideo camera control screen of a video camera controller are the same asin the second and third embodiments, and so FIGS. 19, 20, and 21 areagain used for these parts.

An outline of the operation of a video camera control method accordingto the fourth embodiment will be described with reference to FIG. 27.FIG. 27 shows an operation on the video camera control screen, FIG. 21.Note that an operation when the user selects a video camera movable areaas a new object to be imaged is the same as the operation, FIG. 24A,explained in the third embodiment, and so a detailed description thereofwill be omitted.

FIG. 27 shows an example when the user selects a position outside thevideo movable area as a new object to be sensed. In FIG. 27, it isimpossible to change the direction of a video camera 101 such that aselected coordinate point 130 comes to the center of the screen, sincethe selected coordinate point is outside a video camera movable area 112b. Therefore, a point 131 on the frame of the video camera movable area112 b, which is closest to the selected coordinate point 130, is set asthe center of a new sensing range. However, if the camera direction ischanged without changing the zoom magnification, the selected coordinatepoint 130 sometimes does not come into the sensing range as indicated bythe positional relationship between a sensing range display area 112 dand the selected coordinate point 130. Therefore, the zoom magnificationis minimized (to the widest angle). By the above control, the selectedcoordinate point 130 is included in a sensing range display area 112 e,and this makes scene sensing possible.

Details of the operation of the video camera control method according tothe fourth embodiment will be described below with reference to FIGS. 28and 29. FIG. 28 illustrates a sensing enable area 112 a as a planehaving a size of m×n. Coordinates are written in this plane, and theplane is divided into a plurality of areas. Reference numerals in FIG.28 are the same as in FIG. 27. FIG. 29 is a flow chart showing theoperation of a system controller 105.

The user operates an operation unit 106 to move a cursor 119 in a videocamera control window 10 on a screen 100 a of a display 100, selecting aplace to be sensed (step S200).

The system controller 105 detects the coordinates (Xp,Yp) of theposition of the cursor 119 when the place to be sensed is selected instep S200 (step S201).

Subsequently, the system controller 105 checks which of areas A, B, C,D, E, F, G, H, and I the selected coordinate point 130 (Xp,Yp) belongsto (step S202).

If the system controller 105 determines in step S202 that the selectedcoordinate point (Xp,Yp) is within the area A, i.e., the video cameramovable area 112 b, it is not necessary to change the zoom value.Accordingly, the system controller 105 calculates only the moving amountof the video camera 101 (step S203).

If the system controller 105 determines in step S202 that the selectedcoordinate point 130 (Xp,Yp) is within any of the areas B, C, D, E, F,G, H, and I outside the video camera movable area 112 b, the systemcontroller 105 calculates the coordinates of the center 131 of the newsensing range (step S204). A method of calculating the center 131 of thenew sensing range when it is determined that the selected coordinatepoint 130 (Xp,Yp) is in the area B as illustrated in FIG. 28 is asfollows.

The point 131 on the frame of the video camera movable area 112 b, whichis closest to the selected coordinate point 130 (Xp,Yp), can be easilyobtained by(Xp,Yb)since the point 131 is the intersection of a boundary liney=Ybbetween the area B and the video camera movable area 112 b and aperpendicularx=Xpfrom the selected coordinate point 130 (Xp,Yp) to the straight liney=Yb. That is, when the area B is selected, the X coordinate of the newposition of the video camera 101 can be determined from the selectedcoordinate point 130 (Xp,Yp), and the Y coordinate is unconditionallyYb. Similarly, when the area F is chosen the Y coordinate isunconditionally Yf. When the areas D and H are chosen, the X coordinatesare unconditionally Xd and Xh, respectively, although the Y coordinatesare respective designated values.

When the selected coordinate point 130 belongs to any of the areas C, E,G, and I, the coordinate point at the corner of the video camera movablearea 112 b, which is closest to the selected coordinate point 130, isunconditionally set as the new center point 131 without performing theabove calculations. For example, if the coordinate point (Xp,Yp) iscontained in the area C, a corner point 133 is set as the imagingcenter.

The system controller 105 then minimizes the zoom magnification (to thewidest angle) and calculates the amount of movement to the new centerpoint 131 calculated in step S204 (step S205).

Subsequently, the system controller 105 checks whether the video camera101 as an object to be controlled is connected to a video cameracontroller in a remote place or can be controlled by the controller 105(step S206).

If the system controller 105 determines in step S206 that the videocamera is connected to a video camera controller in a remote place, thesystem controller 105 transmits the video camera control informationcalculated in step S203 or S205 to the video camera controller in theremote place via a communication controller 104 (step S207).

If the system controller 105 determines in step S206 that the videocamera 101 can be controlled by the controller 105, the systemcontroller 105 changes the camera direction and the zoom magnificationof the video camera 101 on the basis of the control information of thevideo camera 101 calculated in step S203 or S205 (step S208).

By repeating the operation from step S200 to step S208 described above,it is possible to take an image of a given object located in the sensingenable range of the video camera 101 as an object to be controlled.

In the processing of this embodiment, interrupt is accepted. Interruptoccurs when the user completes the operation of the system (step S209),and the processing is ended.

In this embodiment as described above, even if a place selected by theuser is outside the video camera movable area, the video camera 101 ismoved to the limit toward the position selected by the user and stoppedat that limiting position, and the zoom magnification of the videocamera 101 is minimized (to the widest angle). Accordingly, noadditional load is applied on the camera driver and an image of theselected place is picked-up (displayed). Consequently, the desire of theuser to see the selected place can be satisfied. Also, the camera cannotbe pointed in an imaging inhibited area.

In the systems of the first to fourth embodiments as described above,even if a place selected by the user is outside the video camera movablearea, the driver for changing the direction of a video camera is notdamaged. Also, the user can limit the camera movable range, and thecamera cannot be pointed in that direction.

Additionally, the user can instantly know that the selection is an erroreven though the video camera does not move.

Furthermore, even if a place selected by the user is outside the videocamera movable area, the video camera is moved to the limit toward theposition selected by the user and stopped at that limiting position, andan image corresponding to the view angle of the video camera isdisplayed. Accordingly, no additional load is applied on the cameradriver and an image of the selected place or of the selected place andits vicinity is picked-up (displayed). Consequently, the desire of theuser to see the selected place can be satisfied.

An image of the selected place can be reliably picked-up (displayed).

An area outside the video camera movable area can be processed as fourportions, and this facilitates the processing.

An area outside the video camera movable area can be divided into aplurality of areas, and each individual area can be unconditionallyprocessed.

Fifth Embodiment

The fifth embodiment of the present invention will be described indetail below with reference to the accompanying drawings. Thisembodiment relates to a technique by which an object to be sensed ischanged without moving a camera in the systems of the first to fourthembodiments.

FIG. 30 is a block diagram showing an outline of the configuration ofthe fifth embodiment of the present invention. Referring to FIG. 30, aCPU 22 for controlling the overall system, a main storage 24 of the CPU22, a bit map display 35, a mouse 28, a network interface 36, and aframe memory 320 are connected to a bus 38.

A video capture unit 326 converts an output video signal from a videocamera 10 into a digital signal and outputs the signal to the framememory 320. An object lens of the video camera 10 is a high-resolution,wide-angle lens capable of picking-up an image of large area within thefield of view of the lens. An image pick-up means of the video camera 10is desirably a Highvision camera or a device with a higher resolution.

FIG. 31 shows an example of the display screen of the display 35. Awindow system capable of displaying a plurality of overlapped windows isoperating on the display 35. In FIG. 31, an image display window 330 andan operation panel 332 are displayed. The image display window 330displays a portion of an image picked-up by the video camera 10. Theoperation panel 332 is used to designate which portion of the inputimage from the video camera 10 is to be displayed on the image displaywindow 330, and to designate the magnification at which the selectedimage portion is displayed. The operation panel 332 includes a button338 for designating an imaging inhibited area and a clear button 339.The operation panel 332 further includes a position designation panel334 and a magnification designation panel 336. The position designationpanel 334 is used to designate a portion of the input image from thevideo camera 10 which is to be displayed in the image display window330. The magnification designation panel 336 is used to designate themagnification of the image displayed in the image display window 330.

In this embodiment, designation of the imaging inhibited area is done inthe same manner as in the fourth embodiment. Therefore, a detaileddescription of a procedure of the designation will be omitted.

FIG. 32 shows the relationship between an image (input image) picked-upby the video camera 10 and stored in the frame memory 320 and an image(display image) displayed in the image display window 330. Referencenumeral 340 denotes an input image, i.e., an image which is picked-up bythe video camera 10 and stored in the frame memory 320 by the videocapture unit 326 and from which an imaging inhibited area is excluded.Reference numeral 342 denotes an area extracted from the input image 340and displayed in the image display window 330. If the extraction area342 is outside the input image 340, fixed-color data or the like isdisplayed in this outside portion. This allows a user to readilyrecognize on the screen that he or she is attempting to see a portionoutside the sensing range or a portion outside the imaging inhibitedarea. Therefore, except in such a case or portion, the extraction area342 generally coincides with the display image.

Assuming that the vertical and horizontal directions on the imagingscreen of the camera 10 are x and y axes, respectively, the angles inthe x and y directions are represented by θ and ø, respectively, andθ=0, ø=0, x=0, and y=0 in the center of the input image, the centralcoordinates (xl,yl) of the display image are represented byxl=L tan θyl=L tan øwhere L is a constant. The display area to be extracted from the inputimage is determined by this origin coordinate point (xl,yl) and thedisplay magnification. The higher the display magnification the smallerthe vertical and horizontal dimensions of the extraction area.

The pixel density of the image extracted from the extraction area 342 isso converted as to meet the number of pixels in the image display window330. Assuming, for example, that the numbers of pixels in the imagedisplay window 330 are H (vertical)×W (horizontal) and the numbers ofpixels in the area extracted from the input image are h (vertical)×w(horizontal), H and W rarely equal h and w, respectively. Generally, thepixel data in the extraction range is interpolated and thinned so as tomeet the number of pixels in the image display area of the image displaywindow 330. Various known methods are usable as this pixel densityconversion method.

The center (xl,yl) of the extraction area 342 can be freely designatedby operating four-direction keys on the position designation panel 334by using the mouse. Also, an arbitrary display magnification can bedesignated on the magnification designation panel 336. The vertical (h)and horizontal (w) dimensions of the extraction range 342 are determinedby the magnification designated on the magnification designation panel336 and the vertical and horizontal dimensions of the image displaywindow 330.

The image in the extraction area 342 is displayed on the display 35 ortransmitted to a remote place via the network interface 36.

In this embodiment, tilt(θ), pan(ø), and zoom(z), the same commands asused in a camera operation command system, are used in a command systemfor changing the center of the extraction area 342. Tilt(θ) is a commandfor tilting in the vertical direction, pan(ø) is a command for panningin the horizontal direction, and zoom(z) is a command for zooming.

That is, the tilt command is assigned to the vertical keys on theposition designation panel 334. While these keys are depressed, the CPU22 increases or decreases the angle θ to generate a tilt(θ) command. Thepan command is assigned to the horizontal keys on the positiondesignation panel 334. While these keys are depressed, the CPU 22increases or decreases the angle ø to generate a pan(ø) command. Thezoom command is assigned to the scroll bar on the magnificationdesignation panel 336. In accordance with the movement of this scrollbar, the CPU 22 generates a zoom(z) command of a magnification z meetingthe movement.

The tilt and pan commands change the center of the area 342 extractedfrom the frame memory 320, and the zoom command changes the size of theextraction area 342. These commands can be transmitted to a remote platevia the network 36. That is, it is possible to remotely change the areaextracted from a frame memory in an apparatus with the sameconfiguration as in FIG. 30 installed in a distant place.

FIG. 33 shows the flow of a procedure from imaging to image display inthis embodiment. A dynamic image can be reproduced and displayed if thisprocedure is executed within one frame cycle of a dynamic image.

Referring to FIG. 33, steps S334 and S335 correspond to a controlprocedure done by a video communication controller 20. When an image isinput from the camera 10 (step S331), the video capture unit 326 writesthe image data into the frame memory 320 (steps S332 and S333). Inaccordance with the tilt angle, pan angle, and zoom magnificationdesignated on the panels 334 and 335, the extraction area 342 isdetermined and the image is extracted as shown in FIG. 32 (step S334).The extracted image is converted to meet the size of the window 330 inwhich the image is to be displayed (step S335). The converted image isthen output and displayed (step S336).

The above procedure is executed for a local camera or by a client in aremote place. To operate a camera in a distant place, the tilt, pan, andzoom commands are generated and transmitted to a controller of thecamera in that place.

If the pan, tilt, and zoom operations are performed for the same camera,it is only necessary to execute steps S334 and S335.

As described above, the display direction and width can be changed byelectronically changing the area 342 extracted from the frame memory320. Accordingly, it is possible to obtain essentially the same effectas when the direction and the zoom magnification of a single camera areinstantaneously changed.

Remote control is made possible by transmitting an image extracted fromthe extraction area 342 of the frame memory 320 to a terminal in adistant place via the network and transmitting the individual commandsdescribed above from the terminal in the distant place. The remotecontrol is basically the same as when panning, tilting, and zooming of acamera are remotely operated.

In this embodiment, an image extracted from the extraction area 342 ofthe frame memory 320 is displayed in the image display window 330 of thebit map display 35 and output to the network where necessary. However,it is obvious that the image can also be written into a file. That is,any output form is usable as long as the processing speed issufficiently high. The image also can be compressed by the JPEG or MPEGcoding scheme before being transferred.

It is also evident that if the video camera 10 includes a memory meanswhich can be externally controlled, this memory means can be usedinstead of the frame memory 320.

Although one extraction area 342 is set for an input image in the aboveembodiment, a plurality of extraction ranges also can be set. If this isthe case, each extraction range is provided with a camera window havingan image display window for displaying an image extracted from theextraction range and an operation panel for setting the extraction area.FIG. 34 shows an example of a screen having four camera windows 350,352, 354, and 356. FIG. 35 shows an example of setting of correspondingextraction areas 358, 360, 362, and 364 in the frame memory 320. Imagesextracted from the extraction areas 358, 360, 362, and 364 are displayedin image display windows 350 a, 352 a, 354 a, and 356 a in the camerawindows 350, 352, 354, and 356, respectively. Operation panels 350 b,352 b, 354 b, and 356 b in the camera windows 350, 352, 354, and 356 areused to operate the positions and sizes of the extraction ranges 358,360, 362, and 364, respectively.

The correspondence between the camera windows 350, 352, 354, and 356 andthe extraction areas 358, 360, 362, and 364 is managed by a table asillustrated in FIG. 36. Virtual camera numbers directly correspond tothe extraction areas 358, 360, 362, and 364. In accordance with anoperation on the operation panels 350 b, 352 b, 354 b, and 356 b, thecorresponding values of θ, ø, and zoom magnification in the row of thecorresponding operation panel ID are updated, and the correspondingextraction area is reset in accordance with these new values.

FIG. 37 is a flow chart of an operation of capable of setting aplurality of extraction areas. This flow chart is formed by extendingthe flow chart shown in FIG. 33 so as to correspond to each entry in thetable in FIG. 36. The flow chart in FIG. 37 differs from the flow chartin FIG. 33 only in that loop processing is added. Assuming the framerate of a dynamic image is F (frame/sec), N images can be extracted froman input image from a single camera so long as a procedure from stepS311 to step S319 is executed within 1/F second. To transfer the imagesto the network, virtual camera numbers are added to the image data to betransmitted in order to be able to identify which image is extractedfrom which extraction area. A plurality of images with different virtualcamera numbers can be displayed as if images from a plurality of camerasare displayed.

In this way it is possible by physically using a single camera to obtainimages in various directions at various zoom magnifications, as if aplurality of cameras were used. It is of course possible toindependently and instantaneously change the directions and zoommagnifications of the individual images.

By the use of a graphical user interface shown in FIG. 38, one or moreextraction areas can be set more visually. Reference numeral 370 denotesan operation window for setting extraction areas. The size of thisoperation window is proportional to the size of an image obtained byexcluding an imaging inhibited area from an input image to the framememory 320. In the window 370, rectangular frames 372, 374, 376, and 378indicating extraction areas with desired positions and sizes are set.These rectangular frames 372 to 378 are expandable and movable withinthe area of the window 370. The position and size of each rectangularframe in the window 370 represent the position and size of thecorresponding extraction area. When the rectangular frames 372 to 378are initially set, the points at opposing corners of each rectangularframe are designated by using, e.g., a mouse. As shown in FIG. 38, aportion of an input image to which each extraction area corresponds tocan be clearly seen.

As can be readily understood from the above description, in thisembodiment the same effect as one obtainable when a plurality of camerasare used can be obtained by physically using a single camera. That is,it is possible to instantaneously change the direction and the zoommagnification of the virtual camera and to simultaneously obtain imagesin various directions at various zoom magnifications.

This embodiment can be combined with any of the second to fourthembodiments.

In a case where the second embodiment is applied, if a designatedextraction frame overlaps an imaging inhibited area, this designation isneglected and an error message is displayed.

In a case where the third or fourth embodiment is applied, if adesignated extraction frame overlaps an imaging inhibited area, theentire extraction frame is moved by an amount corresponding to theoverlapped portion. More specifically, the extraction frame is movedtoward the inside of an input image by amounts corresponding to theoverlapped lengths in both the X- and Y-axis directions.

The systems of the first to fifth embodiments described above can alsobe realized by connecting a camera to a general-purpose apparatus suchas a personal computer and supplying programs for performing the controlin these embodiments from a storage medium.

In this case, the program codes read from the storage medium realize thefunctions according to the embodiments, and the storage medium storingthe program codes constitutes the invention.

Further, the storage medium, such as a floppy disk, a hard disk, anoptical disk, a magneto-optical disk, CD-ROM, CD-R, a magnetic tape, anon-volatile tape memory, and ROM can be used for providing the programcodes.

Furthermore, besides aforesaid functions according to the aboveembodiments are realized by executing the program codes which are readby a computer, the present invention includes a case where an OS(Operating System) or the like working on the computer performs a partor entire processes in accordance with designations of the program codesand realizes functions according to the above embodiments.

Furthermore, the present invention also includes a case where, after theprogram codes read from the storage medium are written in a functionexpansion card which is inserted into the computer or in a memoryprovided in a function expansion unit which is connected to thecomputer, CPU, or the like contained in the function expansion card orunit performs a part or entire process in accordance with designationsof the program codes and realizes functions of the above embodiments.

In a case where the present invention is applied to the aforesaidstorage medium, the storage medium stores program codes corresponding tothe flowcharts described in the embodiments.

1. An image processing apparatus connected view a network with aplurality of external apparatuses each of which has an operation paneland an image displaying window, comprising: a storage unit; and anoutput unit for outputting a plurality of images which are extractedfrom an image stored by said storage unit in accordance with areas setby the operation panels to the image displaying windows corresponding tothe operation panels to display the plurality of images.
 2. The imageprocessing apparatus according to claim 1, wherein the operation panelsets a center position and a size of the area.
 3. The image processingapparatus according to claim 1, wherein said output unit adds anidentifier to each of the plurality of images to be output.
 4. The imageprocessing apparatus according to claim 1, wherein said storage unitstores an image captured by a camera.
 5. The image processing apparatusaccording to claim 1, wherein at least one of the plurality of externalapparatuses has a plurality of the image displaying windows and saidoutput unit outputs the plurality of extracted images to said at leastone of the plurality of external apparatuses.
 6. An image output methodfor outputting images via a network to a plurality of externalapparatuses each of which has an operation panel and an image displayingwindow, said method comprising: a storing step of storing an image to astorage; and an outputting step of outputting a plurality of imageswhich are extracted from an image stored in said storage in accordancewith areas set by the operation panels to the image displaying windowscorresponding to the operation panels to display the plurality ofimages.
 7. The image output method according to claim 6, wherein theoperation panel sets a center position and a size of the area.
 8. Theimage output method according to claim 6, wherein said outputting stepadds an identifier to each of the plurality of images to be output. 9.The image output method according to claim 6, wherein said storing stepstores an image captured by a camera.
 10. The image output methodaccording to claim 6, wherein at least one of the plurality of externalapparatuses has a plurality of the image displaying windows and saidoutput step outputs the plurality of images to said at least one of theplurality of external apparatuses.